Systems interacting with the real world usually take noisy measurements of relevant variables, try to estimate the true values of these variables, and make decisions based on the estimates, as well as available statistical, physical and logical models. A graphical representation of a typical system is shown in FIG. 1. As shown in this figure, the estimator 11 combines noisy measurements from sensors 10 with the information available from the prior iterations of the system's execution and statistical information about the measurement noise in order to come up with the estimates of the true values of the measured variables. The decision maker 12 uses these estimates and physical models to make decisions.
We refer to any quantities of interest to the system as state variables. For typical applications these variables may be related to the environment, the system, the system's tasks, and any other cooperating and competing systems. All possible values for state variables form the state space. The system is able to take measurements of some of the state variables using physical and/or virtual sensors. Such measurements are usually corrupted by noise. Estimation is the process of selecting optimal values for measured state variables utilizing measurements, statistical information about noise, any prior statistical information about state variables, and any restrictions on possible values of the state variables. Although estimation may be a system's sole task, often estimates are utilized in further decision making. A system's decisions may affect a subset of state variables. Decisions are made to accomplish tasks. Tasks may be specified by constraints on the state space that define target Subsets.
Such systems have a wide variety of applications. Two representative examples are controllers and prognostic systems. In the first example, a controller may generate control policies for an autonomous robotic system. The system uses its sensors to take noisy measurements of the environmental and system's state variables. The controller computes the control law to be used over the next control loop iteration based on these measurements, previous state estimates, and models. In the second example, the system monitors parts (e.g. gears, actuators, pumps, etc.) by taking measurements of their states and reasoning about their remaining useful life and any required maintenance actions.